Papers with real-world predictions
Unlearning Bias in Language Models by Partitioning Gradients (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent research has shown that large-scale pretrained language models exhibit issues relating to racism, sexism, religion bias, and toxicity in general. |
| Approach: | They propose a gray-box method for debiasing pretrained masked language models using partitioned contrastive gradient unlearning (PCGU) aims to optimize only the weights that contribute most to a specific domain of bias by computing a first-order approximation based on the gradients of contrastive sentence pairs. |
| Outcome: | The proposed method is low-cost and can pinpoint the sources of social bias in large pretrained language models. |